FETCH-O-MATIC by Emmett Gray

Version 2.0.3
Copyright ©1997 Webcast Multimedia, Inc. All rights reserved.
Fetch is a registered trademark of Dartmouth College.

Contents:

Introduction
1. Installation
     A. Quick Start
     B. Detailed Installation Instructions
     C. PPP Note
2. Settings
     A. The "Settings" file
          i. Settings file entries
          ii. Other related issues
          iii. Managing multiple sites
     B. FetchProfile
3. Methods
4. Operation
     A. General
     B. FetchNewLocal:
          i. What It Does
          ii. The "To Upload" file
     C. FetchUp
          i. What It Does
          ii. The changesFile
          iii. Interrupted Uploads: FetchUp Auto-Correction
          iv. The messageFile
          v. The "Activity Log" file
          vi. Simultaneous Uploads
     D. FetchChanges
     E. FetchDown & The "To Download" file
     F. FetchTimeStamp, FetchMe
5. The Utilities
     A. FetchCompare
     B. FetchDelete
     C. FetchMeNot
     D. FetchStructure
6. Miscellaneous
     A. "Secret" Files/Areas: Multiple Fetch-O-Matic Suites
     B. Stopping Execution
     C. Error Messages
         "Apple event timed out"
         "Couldn't find..."
         "Error number..."
         "Local changesFile has data not present in the remote file..."
         "Not connected"
         "Out of memory"
         "That file or directory is unavailable or nonexistent"
         "This file may be nonexistent"
         "Tried to create temporary file..."
         "Unable to connect to remote server"
         "Unexpected server response"
         "Unix or DOS Linefeeds..."
         "You must install Remote Access"
         "<<script>> doesn't understand the <<event xxxx>> message"
         Other (strange-looking) AppleScript errors
     D. Memory Allocation
     E. Fetch Preferences
7. Frequently Asked Questions
8. Obtaining Software
9. Registration and Payment
10. Legal Notice
Appendix A: List Files settings
Appendix B: Required Scripting Additions
Appendix C: Revision History, Updating Notes
Appendix D: Credits and Thanks
 

INTRODUCTION

Fetch-O-Matic is a new approach to automated website maintenance via FTP. Fetch-O-Matic allows web authors to freely create and update files locally without needing to keep track of their work, and then automatically catalog the alterations and update the website.

Fetch-O-Matic differs from other available synchronization and maintenance schemes in that it does not require any local machine to have a complete site copy. For example, one might decide to grab a few pages and change them (and/or add some new files) from a PowerBook. Later, the site copy on the primary machine can be quickly brought up-to-date with just two clicks - without having to remember what was changed or open the PowerBook.

Fetch-O-Matic is extremely easy to use. Novice web authors in search of a simple way to update a personal web page won't be intimidated by its power. Experienced hands will find that previously dreaded operations such as multi-file search and replace, with the subsequent headache of tracking down and uploading scattered altered files, are rendered painless. Transfer errors such as uploading/downloading to the wrong directory are a thing of the past. If several people are maintaining a site they are protected from treading on each other's updates.

Fetch-O-Matic was conceived and developed to fill a need in servicing a large site (http://www.filmscouts.com). Uploads and downloads of thousands of files at once have been performed with ease. But even on a site with just a few pages, Fetch-O-Matic will ease the updating chores.

Fetch-O-Matic requires Dartmouth's FTP client "Fetch", AppleScript, the scripting addition Jon's Commands, and, optionally, Alessandro Levi Montalcini's "List Files" (providing a significant speed improvement for large sites). It runs on any Mac OS machine with reasonable (8mb+) RAM running System 7 or 8. It runs in the background. Fetch-O-Matic is shareware with a single-user license fee of $20. There is a free 30-day trial period.

Back to Table of Contents


1. Installation.

A. Quick Start

1. You must have "Fetch" installed, and, optionally, List Files. The AppleScript extension and Apple's standard Scripting Additions are also required. Also required is a text editor or word processor of your choice.
2. Put Jon's Commands, Password, setDate, and perhaps a PPP component (see PPP note) in your Scripting Additions folder.
3. Open, configure, and save the "Fetch-O-Matic Settings" file; instructions are in that file.
4. Run "FetchProfile" and do a "Full" test (if using List Files, also select the option to set List Files prefs, then follow the presented instructions).
5. If FetchProfile's result file ("F-O-M Profile...") reports only a single problem - a missing "<sitename> filelist" file, normal at this point - you are ready to use Fetch-O-Matic.

B. Detailed Installation Instructions

The folder containing the Fetch-O-Matic suite may be placed anywhere on your hard drive. The suite contains the following files: the applications FetchChanges, FetchDown, FetchMe, FetchNewLocal, FetchTimeStamp, and FetchUp; the text files Activity Log, To Download, To Upload; the "Fetch-O-Matic Auxiliary" folder containing the Register application, the text files Fetch-O-Matic README, Fetch-O-Matic Settings, and this User Guide, a folder "User Guide Illustrations" accompanying this Guide, a file "List Files Prefs", a folder "Stuff for Scriptg. Addns." containing the AppleScript scripting additions FreePPP Control (by Sebastian Kloska), MacPPP Control (by Mark Alldritt), PassWord (by Donald Olson), and setDate (by Mikael Flensborg); the folder "Test & Doc - FreePPP Ctrl" with a test application and ReadMe file re FreePPP Control, and the "Fetch-O-Matic Utilities" folder containing the applications FetchCompare, FetchDelete, FetchMeNot, FetchProfile, and FetchStructure. (Additional files will appear in both the Auxiliary and Utilities folders as various applications are run).

The AppleScript extension (1.1 or later) must be installed, along with the standard Apple scripting additions, in order to use Fetch-O-Matic. (AppleScript and the additions are included with Apple system software 7.5 and later. The latest version can be found at http://applescript.apple.com/). Fetch-O-Matic is fully compatible with all known versions of Mac OS System 7 and 8 and should run on any Mac OS machine.

You will need to get three additional widely available files before you can use Fetch-O-Matic: a) "Fetch" 2.7 or later (3.0.3 is current as of this writing), by Jim Matthews/Dartmouth College ($25.00 registration fee; free if you are part of an educational or charitable organization); b) "Jon's Commands" 1.8 or later (2.01 current), by Jon Pugh (free for private use; donations accepted. Note: type "jons", not "Jon's", when searching for this file); and c) "List Files" 2.6 or later, by Alessandro Levi Montalcini ($10.00 registration fee). (Having and using "List Files" is not required, but it speeds things up and is recommended). All three can be found on the Fetch-O-Matic download page (http://www.filmscouts.com/software/fomdown.html), Info-Mac (http://hyperarchive.lcs.mit.edu/HyperArchive/SearchForm.html), Shareware.com (http://www.shareware.com/), and many other Mac http and ftp software sites.

After getting these files, if you are using System 7.x then put Jon's Commands, PassWord, setDate, and perhaps MacPPP Control or FreePPP Control (see PPP Note) in the Scripting Additions folder in the Extensions folder in your System folder; under System 8, just drag these to the closed System folder and they will be correctly installed (the Scripting Additions folder has been "promoted" to System folder root level under System 8). Install Fetch and List Files anywhere on your hard drive.

Fetch must be configured in its Preferences under "Download" to overwrite files with conflicting names, and should otherwise be configured normally for use on your site. You don't need to create any bookmarks or enter passwords since the signon process is handled by Fetch-O-Matic. Secure FTP access (via SKey and other one-time password schemes) is automatically supported directly by Fetch 3.0 and later, and thus compatible with Fetch-O-Matic as well. See more information below under Fetch Preferences.

C. PPP Note:

If you are using Open Transport, you don't need the either of the scripting additions "MacPPP Control" or "FreePPP Control" and should discard them.

If using Open Transport with OT/PPP, the OT/PPP installer will have placed the Scripting Addition "PPP Commands" in your Scripting Additions folder. If so, you may select the "O" option on the "usePPP" line in the Settings file and enable the potential for Fetch-O-Matic to close a connection (hang up) on terminating. The terminating behavior is set in the "closeLink" line and is described in the Settings file. (Whether you have this Scripting Addition or not, Fetch-O-Matic can make connections automatically). If you do not have "PPP Commands", get a more recent version of OT/PPP, with which it is bundled. This is available for download from official Apple software sites.

MacPPP Control offers the same functionality as PPP Commands, but works only with MacTCP and not with Open Transport. It works with versions of MacPPP and FreePPP up to and including version 2.5v2, but not beyond. (MacPPP 2.5 can be found on the various official Apple software sites). If using this generation of PPP software, set the "usePPP" line in the Settings file to "M" and install this addition.

FreePPP Control is similar and works with FreePPP 2.6 and later. If using this generation of PPP software, set the "usePPP" line in the Settings file to "F" and install this addition. (The folder "Test & Doc - FreePPP Ctrl" contains the original documentation and a test application for this Scripting Addition. This is a beta release but has been tested with Fetch-O-Matic without problems).

Back to Table of Contents


2. Settings.

A. The Settings File.

i. Settings File entries

In the "Fetch-O-Matic Auxiliary" folder there is a text file named "Fetch-O-Matic Settings". This file must reside in the Auxiliary folder, which must remain within the main Fetch-O-Matic folder, and you may not change the names of the Auxiliary folder or the Settings file.

The Settings file is where data shared by all the Fetch-O-Matic applications is stored. Open this file with a text editor or word processor and enter the required information, replacing the dummy values, and then save the file. Instructions for entering the information are contained in the Settings file itself. Once you have done this, run the FetchProfile application, which will check your settings and create necessary files.

The password setting has two lines, one for a password-saving preference and one for your FTP logon password. There are three options, with convenience inversely proportional to security: if you elect to enter your password here your security level is zero, but you won't be bothered by any dialog boxes. The middle ground, selecting a letter other than "N" for the passWordSave option, will store your password (encrypted) and you will be asked only once for it by each Fetch-O-Matic application that needs it; however, anyone with access to your machine can log on - although this won't enable them to know your password. The only secure option is to set the passWordSave option to "N" and enter nothing (other than spaces) in the passWord line. But then you will then be queried for your password every time you run an application that needs it. Further instructions (i.e. regarding changing your password if stored) are in the Settings file.

There are two more text files which must exist, referred to in the Settings file (and in the following discussion) as the "changesFile" and "messageFile". These may be renamed and/or moved, but must exist WITHIN YOUR SITE on both local and remote machines. (You may use the default names for these files as they appear in the Settings file, which will place them at the root level of your site). The FetchProfile application will automatically create these files (if they don't already exist). These files are used by Fetch-O-Matic to keep a record of what activity has taken place on the remote host. The changesFile is written to automatically. The messageFile is just a note pad which you can use as you see fit, and isn't consulted about action to take, but if the remote copy has changed you are notified and the updated version is downloaded and opened - an automatic alert.

The "reconnect" setting permits you do enable or disable an attempt by Fetch-O-Matic to automatically resume an interrupted transfer session, due to a dropped connection or similar error. If reconnection is successful, any file partially uploaded or downloaded will be resumed from the beginning, and the session will then proceed with remaining files in the queue. Although most users will want to allow this, the option to disable is provided since there may be circumstances, such as downloading huge files, for which reconnection is not desired, since it will conflict with Fetch's potential to (manually) resume an interrupted download of a large file. See the Fetch Help topic "Resume Download" for more information on Fetch's resumption capabilities.

The "ignoreFolder" setting permits the identification of a single folder in your local drive whose contents, including any subfolders and their contents, will not get added to the To Upload list by FetchNewLocal. This can be useful for work-in-progress or for template storage, permitting, for example, a local page in development to have correct relative links to graphics etc.

Due to the many possible situations for its use, Fetch-O-Matic does not attempt to deal with Fetch's preferences, except as noted below under Fetch Preferences.

ii. Other Related issues

The text files "Activity Log", "To Download", and "To Upload", must be in the main Fetch-O-Matic folder and may not be renamed. These are empty files to start with, but they must exist.

In the Auxiliary folder, there will be a "<sitename>filelist" file created by either FetchNewLocal or List Files (if used), whose name will be determined by the name of your local site copy (the "localHome" setting). (If using List Files, you must configure it to save its output file in the Auxiliary folder. This should be done by running the FetchProfile application, which simplifies the setup of List Files). This file (and some others) won't exist until you run the application which creates it. When you run FetchProfile before running FetchNewLocal (as you should do), it will fail to find this file, which doesn't yet exist. This error is normal.

In order for the FetchNewLocal application to work with List Files, you must first set List Files up with the correct preferences. The FetchProfile application will take care of this for you, and will prompt for the destination folder of the output file, and that file's creator application (text editor or word processor). This output file must be saved in the "Fetch-O-Matic Auxiliary" folder. To set this option, select "File Saving... "under the "Options" menu in List Files and navigate to the correct destination. There is also an appendix at the end of this document with a series of illustrations of the proper settings for the various List Files options. Your set up as should be illustrated, and also with a checkmark next to the menu item "Skip Save Warnings" under the "Options" menu.

The file "List Files Prefs", in the Auxiliary folder, should not be moved. Each Fetch-O-Matic folder maintains its own copy of the List Files preferences file so as to allow you to use List Files for other things, including running other copies of Fetch-O-Matic for different sites. These preferences are swapped out with the one in the System folder's preferences folder each time FetchNewLocal or FetchCompare runs. If you are already using List Files, as long as you use FetchProfile to set the preferences for Fetch-O-Matic, your previous preferences will be maintained. If you have not used List Files before but now wish to use it independently of Fetch-O-Matic, you can set up any other preferences you like and these will be preserved by Fetch-O-Matic.

iii. Managing Multiple Sites

If you are managing multiple sites you must keep a Fetch-O-Matic folder for each site. Each Fetch-O-Matic folder must contain the applications (not aliases) and a Settings file, and must of course relate to a separate complete set of Fetch-O-Matic text files as well. (You may of course use aliases to any of the Fetch-O-Matic applications or files). For managing separate areas of a single site, see the section "Secret Files/Areas".

 

Back to Table of Contents


B. FetchProfile.

FetchProfile eases the setup process by checking your settings and creating needed files. It does a dry run with the configuration in your Settings file, and looks for problems such as missing files, applications, and Scripting Additions. It will create the local and remote changesFile and messageFile if they don't already exist. It also leads you through the setup of List Files preferences, if you are using List Files.

You should run this application after entering your settings and before you attempt to run any other Fetch-O-Matic application. At this time it will report a missing "<sitename> filelist" file, which is normal. After you have run FetchNewLocal, run FetchProfile again, and you should get a report stating "No problems found".

You should also run FetchProfile again after installing List Files if you do that at a later time. You should run this application whenever you want to check your current settings, such as when running for the first time from another machine.

FetchProfile offers options on launch. It then proceeds with the selected actions and creates a file with title "F-O-M Profile <date>" in the Fetch-O-Matic Utilities folder. This will show what it did and what it found. This result file is not used subsequently and may be discarded.

Back to Table of Contents


3. Methods.

Fetch-O-Matic operates by automatically generating lists of files requiring action and storing these lists in the "To Upload" and "To Download" files. Whether uploading or downloading, Fetch-O-Matic operates by duplicating whatever file structures it sees in the paths of the items in these files. It will create any elements of that path that don't already exist. So, for example, on your local machine you could decide to create a new folder and put in it a bunch of other files and folders, to any depth you like. All of this structure will be transferred to the site, and downloaded intact to any other machine running Fetch-O-Matic. When building your site, you may simply forget about structural transfer issues and put things together any way you want, creating files and folders at will all over your local site copy.

Because Fetch-O-Matic operates by consulting a central logfile (the changesFile) which contains only paths and filenames, no machine involved in site work needs to have a complete copy of the site. The only requirement for every machine doing work is that the DIRECTORY STRUCTURE encompassing every FILE being worked on is complete, and that copies of the changesFile and the messageFile exist in their proper locations (and are current) when work is begun. In other words, all the paths to files must be exactly as they are (or will be) be on the main site; every level in the path from the file in question to the root of the main site must be present on the copy. Items - files and folders - outside this direct line to the root may be ignored or dispensed with in the site copies as desired. If you are starting work from a new location that has no site copy, at the very least you need to create a local root directory, have a Settings file with correct information for that machine, and run FetchProfile to create the local changesFile and messageFile. If you then create files from scratch on this machine, you must also create locally their enclosing directory structure, if any. You may also run FetchChanges at this point, which, since there are no files in your local root, will put every recently uploaded file into your To Download List, which you may then edit as needed. Finally, if you wish to create a (more) complete local mirror, you may run FetchCompare, which will create a To Download list which represents the entire remote site.

Because Fetch-O-Matic overwrites files without warning, you must be sure that the relevant local and remote structures match. The author recommends testing Fetch-O-Matic by creating a root-level dummy "test" folder on your local machine with several levels and dummy files and directories, and verifying operations with that until you are comfortable with the results. Create test files, run FetchNewLocal, run FetchUp. Then remove the local files, remove the added lines from your local changesFile (as if you were a different machine with no knowledge of those files), run FetchChanges and then FetchDown. Everything you had removed should reappear. If this doesn't happen then there is an error in your configuration of Fetch-O-Matic's settings.

If you create empty directories on your local machine, including nested ones, they will not be uploaded. Fetch-O-Matic only deals with files. It actually doesn't upload or download directories at all; rather, when it begins to process a file, it first looks for parent directories and only then, if they don't already exist, creates as many of them as needed to contain the files inside them. Thus there must be at least one file in a directory (or one of its subdirectories) for Fetch-O-Matic to recognize a need for that directory and do something about it.

Once set up, Fetch-O-Matic is extremely easy to use. Work on your site, run FetchNewLocal (and/or FetchMe, see below), peek at the "To Upload" file and alter it if you want, run FetchUp. You're done. If others (and/or yourself from another machine) are involved, just run FetchChanges (and FetchDown if required) before you start work. Run FetchUp without uploading files - or perhaps just uploading the messageFile - to post a notice informing others you're currently working.

All Fetch-O-Matic applications run in the background, and will allow you to perform other Internet activities like web browsing and email, and you can even open another simultaneous FTP connection with Fetch (but only to a different server - if you open a second Fetch window to your own server, Fetch will get confused because the windows will have the same name, so this is not advisable).

None of the Fetch-O-Matic applications have menus or allow user interaction, other than responses to dialogs. All of them will quit running automatically when their actions are complete. Most of them conclude with an informative dialog which will wait indefinitely for your attention, then cause the application to quit when OK'd. Error messages will also appear in dialog boxes and will cause the application to quit when OK'd. Error messages are also written to the Activity Log.

Fetch-O-Matic was written in AppleScript. The beauty of AppleScript is that existing applications can be integrated, alleviating software creators from wasteful reinvention. The applications "Fetch" and "List Files" are used by Fetch-O-Matic in this way to do the tasks they were designed for, and which they do superbly.

Back to Table of Contents


4. Operation.

A. General.

Before you attempt to run any Fetch-O-Matic applications, you must configure the Settings file, see Settings, above. You should then run the application FetchProfile which will check your settings, create needed files, perform a test log-on, set List File's preferences (if you are using List Files), and finally, produce a report.

The first time you run each Fetch-O-Matic application, it may ask you to locate the applications it needs to communicate with: Fetch, your text application, and List Files (if used). When asked to locate an application, you will be presented with a standard file dialog box. Navigate to the folder where the application is installed and select it. If the name does not match exactly the name as specified in the Settings file, the Fetch-O-Matic application will still run, but you will be asked again the next time you run it to locate the application whose name did not match. If you are repeatedly asked to locate an application, it means that the names do not match exactly. Unlike most Mac file actions, this match is case sensitive, so exact capitalization is required.

Back to Table of Contents

B. FetchNewLocal.

i. What It Does

After configuring your Settings file, run this application first. FetchNewLocal determines, by examining modification dates, what files have changed and need to be uploaded. As an option it employs the application "List Files" to compile a local site file list. "List Files" is significantly faster than the internal mechanism.

The first time you run FetchNewLocal it will show a dummy date. Tell it to "Set Time Only", which will cause it to quit. Then run it again. The displayed time will now be the time when you clicked "Set Time", in YY/MM/DD HH:MM format. As a test, you should now change this to some time prior to when you last updated a file in your local site copy, and allow it to continue to run.

When FetchNewLocal concludes its run it should have identified your recently altered file(s) and listed them in the "To Upload" file, and opened this file. If this did not happen then there is an error in your configuration. Make sure the entries in your Settings file are correct. (At this point, you will also find a complete local site file list in the Auxiliary folder, which you might wish to examine at some point. This file is overwritten each time you run FetchNewLocal, so if you wish to preserve it, you must remove it from the Auxiliary folder; if not found there, it will be created anew each time).

If there is more than one local machine building your site, you will be downloading files created on those other machines. Every time you download files, you should run FetchNewLocal in Set Time mode right afterwards, so that these files will not be seen as new the next time you run FetchNewLocal.

Whenever you do any work on your local site copy, run FetchNewLocal to generate a list of anything that has changed since the last time it was run. This list of changed files is automatically placed at the top of your "To Upload" file, with a blank line separating it from anything already there. Nothing is removed from this file by FetchNewLocal; it only adds to what is already there; thus you can run it many times, building up a larger list of files to upload with each run.

FetchNewLocal resets its internal time stamp every time it runs, and shows you this time when you boot it up. You may change this time if you want to look further back (for example if you think you might have missed something), or less far back (if you have been running FetchMe instead, see below).

ii. The "To Upload" file

The "To Upload" file is created in reverse alphabetical order by directory; files will be uploaded in the order they appear in it. Files listed after a blank line are ignored. On the site for which Fetch-O-Matic was developed, most graphics files happen to be stored in directories whose names start with "z" (as a way of keeping them grouped), and since one will want to upload graphics before HTML pages (so they won't be missing when called), this was a sensible approach. In any case, if you are building a "live" site you should always check the "To Upload" file to see that things are going to be uploaded in a logical order, and make alterations as required.

Both FetchNewLocal and FetchMe write to the "To Upload" file. You may run either application at any time, and you may also enter filenames in the "To Upload" file by typing or pasting. Doing so might be useful in situation where a set of files needs to be re-uploaded on a regular basis. In this case, just keep a list of these files permanently in your "To Upload" file, stored after a blank line. Then copy and paste the list to the top of the "To Upload" file whenever needed.

An option in the Settings file selects whether or not you want to have successfully uploaded files removed automatically from the "To Upload" file by FetchUp. This is a timesaving convenience and is enabled by default, but if desired you may disable it. Since the changesFile will always have a list of the files which you attempted to upload, cleaning out the "To Upload" file by hand is really not necessary for verification purposes.

In the Settings file, you have an option to specify an "ignoreFolder", a folder within the local site whose contents will not get added to the To Upload file by FetchNewLocal.

Back to Table of Contents

C. FetchUp.

i. What It Does.

FetchUp uses the "To Upload" file to determine what to upload. Before running FetchUp, you may manually reorganize the "To Upload" file as you see fit in terms of upload order and postponement. FetchUp will ignore anything after a blank line, so to postpone, just insert a blank line before the files you want to skip. A blank line as the first line will result in nothing uploaded at all. If you have run FetchNewLocal multiple times without uploading, there will be blank lines in it (representing each time FetchNewLocal was run), which you will want to remove.

FetchUp has two main functions: it posts notices about activity and it uploads files. The notices it posts are contained in the changesFile, which is the core of the multi-user aspect of the system. Assuming there's not a conflict (see about the changesFile below), the first thing FetchUp will do is to post a notice that "you" (as defined in your Settings file) are doing something. If you are uploading files, it will post a list of files being attempted. If you are not uploading files, it will simply announce that you are "at work"; this is to avoid the conflict of others working on the same files at the same time. You can make use of the messageFile to announce what (areas or) files you are tackling and tell others to keep out, or just tell them to contact you. Anyone blindly attempting an upload while your "at work" notice is up will be stopped, as described below. This does not outlaw simultaneous uploads, but forces everyone to examine the situation for possible conflicts.

FetchUp always posts the date, an operator's name, uploads being attempted, and successful uploads. The optional parameters are: "at work" or "off" (in the case of nothing currently being uploaded), and "off when finished" or "still working when finished" (posted during an upload in progress). The options are set by default as follows: if nothing is being uploaded, an "at work" message is posted. If something is being uploaded, an "off when finished" is posted during the upload, and "off" is posted when complete.

All possible options can be accessed by holding down the command key during launch. You must hold down the command key until you hear a beep; unfortunately, AppleScript cannot recognize a key press until well into execution, so it may be several seconds before you hear the acknowledging beep. You will then see a dialog box. You may change your name (a temporary change valid for the current run only); post an "off" notice with nothing being uploaded (canceling a prior "at work" notice); post "still working when finished" during an upload and "at work" on completion; or simply cancel execution. The slow response of the command key is actually a benefit here - you have a second to press the key if you launch FetchUp by mistake and decide you want to cancel.

A second dialog will appear which asks if you want to override your setting for behavior on completion of the upload. This dialog can also be accessed by command-launch of FetchChanges and FetchDown.

Back to Table of Contents

ii. The changesFile.

The changesFile on the remote machine is always assumed to be the latest version of that file. So, before FetchUp does anything at all, it downloads the remote changesFile and compares it to your local copy. If the remote version is not the same as yours, FetchUp stops what it was about to do, and instead operates as if it were FetchChanges (see below) - it moves your local changesFile to the trash with suffix .old, and adds any new files it finds for downloading to your "To Download" file.

If your local changesFile is an empty file, as will be the case when first starting or when operating from a new location, the entire contents of the remote changesFile will be added to your "To Download" file. However, if your local file is not empty AND its first line is NOT found inside the remote file, as it should be, then FetchUp assumes that some kind of serious error has occurred and quits. This situation is one which you will have to resolve manually before proceeding further. If your local changesFile is corrupted, use the downloaded remote version as your new local copy and simply delete the lines in it which contain activity you are missing. Then run FetchChanges to get back in sync. If you are operating from a new location but don't want the entire remote changesFile added to your downloadList, then download the remote file manually, and delete lines as above.

Otherwise, if all is well, your current activity - a "working" notice and/or upload attempt - is written to the changesFile and it is uploaded. If there are files to upload they are then uploaded. On completion of uploads, the changesFile is rewritten and uploaded again, and FetchUp quits with a dialog box announcing that it was successful. It also alerts you to any other activity that may have occurred during your upload - see below under "Simultaneous Uploads." (If you have disabled the auto-remove feature, you'll have to manually clean out the uploaded files from your "To Upload" file).

It is possible, and sometimes necessary, to manually write to the changesFile, then manually upload it. This topic is discussed under the "Activity Log" file heading below. The main reason to do this is to erase some action which occurred by mistake, such as uploading a file or files unintentionally. So as not to propagate this error further, you will want to remove all mention of the erroneous files from the changesFile. The only way to do this is by hand, since if you alter your local changesFile and then try to run FetchUp, it will see your file as the older one (not matching the site copy) and move it to the trash. The correct procedure is to simply delete the erroneous upload from the local changesFile and then upload that file manually. If your entire upload session was erroneous, you can simply delete the whole event from the changesFile (restoring it to its status before you did anything). Finally, attend manually to whatever you have to do to put the site back in order.

It will also be necessary to do a manual correction and upload after a broken upload if the automatic recovery procedure of FetchUp (see below) should fail for some reason, such as a corrupt local changesFile.

The changesFile will grow indefinitely. At some point you will want to reduce its size. You may do this at any time, simply by truncating (from bottom upwards, of course) your local copy, then running FetchUp. You should preserve this old information for your records, saving the excised section into a new file. The only thing to watch out for when truncating is the possibility that you will be removing a portion which someone else may not have yet seen, so be sure at least to post a notice in the messageFile that you have truncated the changesFile as of a certain date. Anyone needing a list of older uploaded files can then get it from you.

A settings option will enable reminders when the file grows beyond a selected size.

Back to Table of Contents

iii. Interrupted Uploads: FetchUp Auto-Correction

In the event of a crash, broken connection, or other disturbance of the upload, FetchUp has an automatic recovery procedure. You should just run it again, without altering any of the Fetch-O-Matic text files. It will recognize that an upload was interrupted, and it won't upload anything on this run. First it will check if other activity has taken place on the site. (If someone else has done something in the meantime, that information is handled exactly as under normal circumstances, so you don't need to worry that any other information will be erased or skipped by the auto-correction procedure). Then it will write the successful portion of your upload to the changesFile and re-upload that file, merely putting the changesFile back into synch with what has actually occurred. Then it will quit, allowing you to decide when to resume the interrupted upload. The changesFile will now have the notation "Auto-Correction of this file" above the added files.

This feature is independent of the "reconnect" feature and setting, which attempts to continue an interrupted upload or download session, and which, if successful, will not require FetchUp auto-correction. The reconnect process is transparent and will generate no error message or cause itself to be noticed if successful.

iv. The messageFile

To use the messageFile, just write whatever message you want into it. Although it must exist, its use is optional. In order to upload it, its name must be in your "To Upload" file. This will happen automatically if you run FetchNewLocal or drop the messageFile on FetchMe (see below), or, once you are comfortable operating Fetch-O-Matic, you may simply type its name (and path if any) into the "To Upload" file.

The messageFile is the only file besides the changesFile which is automatically downloaded (rather than being added to the "To Download" file) by FetchUp, FetchChanges, and FetchDown. If you have written something to this file and then run FetchChanges or FetchUp and the remote messageFile has been changed, your local messageFile will be moved to the trash - simply because it is not the same as the remote version, even though it may in fact be newer. (This is done as a safeguard that no messages posted on the remote machine are ever skipped or mistakenly overwritten). If this happens, just open the trash and drag your messageFile (now with suffix ".old") back out, recover your text, and paste it into the new messageFile.

Strategies for use of the messageFile depend on how many people are involved. One method is to always put new messages at the top of the file so as to maintain a message history, and then periodically truncate the file by mutual consent, as with the changesFile. On the other hand, if just two people are involved, messages can be erased when read by the second person. Another idea would be to have people post their initials in the file acknowledging that they've seen a particular message; once everyone has seen it, it can be deleted. If only one person is doing site maintenance, the file may still be useful as a central "to do" list for pending work.

v. The "Activity Log" file.

This file is a record of local activity. It is written to after each individual file transfer in either direction. Error messages are also logged here. Its primary purpose is verification of transfers and disaster recovery - if you get dumped off or have a crash, the log will be the guide as to where and how to resume. FetchUp reads the "Activity Log" file when it sees that a transfer had been interrupted, and FetchUp should be able to automatically recover, but not every circumstance can be foreseen and it is conceivable that it will be necessary to manually copy the "Activity Log" file's list of successfully uploaded files to the changesFile and manually upload that file.

If you must copy files from the "Activity Log" file to the changesFile, you need precision only in the file names, which you should copy and paste exactly as they appear in the "Activity Log" file. You don't need to worry about other details, like the date, time, attempted files, your name, and so on. Such material is posted with an informational purpose alone and is NOT used by the automatic process to create the "To Download" file list, or as a basis for determining what has changed on the site. The fact that the changesFile is DIFFERENT from local copies is sufficient to trigger the automatic list assembly. When reading the changesFile, the file-gathering process looks only for lines without commas to build a file list. All of the informational lines, including attempted uploads, have commas. All such lines (and blank lines) are ignored, except for that one line which exactly matches the first line of the changesFile on the local machine, which represents the demarcation line for new material. (If there is no such line, unless the local changesFile is blank, Fetch-O-Matic will refuse to execute further since things are out of whack and some manual intervention is required).

The "Activity Log" file, like the changesFile, will grow indefinitely. As with the changesFile, you may truncate it at any time. A settings option will enable reminders when the file gets big.

vi. Simultaneous Uploads.

Another person working on the site, either by mutual agreement or by simply looking at the files currently being uploaded as listed in the current changesFile, may decide to simultaneously upload files. This is not a problem. If he runs FetchChanges first, putting his changesFile in sync, then his FetchUp won't complain about your upload currently in progress. However, you won't be aware of his new activity, so to handle this situation, FetchUp looks again at the changesFile whenever it concludes an upload, and if it sees that someone else has logged on in the meantime, it runs as a modified version of FetchChanges (see below). If the other upload has concluded, FetchUp will build a "To Download" file list just like FetchChanges; otherwise, it will show an alert that someone else is now at work, and your changesFile will list the files in progress from his end just as if you had not been on at all. Since the changesFiles is in reverse chronological order, the other person's upload will appear below the successful list from your end, which is more recent because it concluded after he began.

Back to Table of Contents

D. FetchChanges.

FetchChanges snoops the remote changesFile for activity and adds to your "To Download" file if there have been any uploads. To determine activity, FetchChanges downloads the remote changesFile and compares it to your local version (see changesFile, above). The comparison is of the file's actual contents and not of dates (either modification dates, upload dates, or those shown in the first line of the changesFile). If it finds differences it adds all appropriate files to your "To Download" file, just as FetchNewLocal does with your "To Upload" file. It doesn't erase anything from this file, and adds at the top of the file. You can run FetchChanges multiple times without downloading anything and the list will accumulate in the "To Download" file, with a blank line inserted after each FetchChanges run.

After downloading the changesFile and comparing it to your local copy, if there is a difference your old changesFile is moved to the Trash with extension ".old" after the name. If there is no difference, no further action occurs. If FetchChanges detects that the messageFile has been changed it downloads the new messageFile (and puts the old one in the Trash with extension ".old"). After opening any new files, FetchChanges quits with an informational message.

If the command key is pressed at (or shortly after) launch, a dialog will appear which asks if you want to override your setting for behavior on completion of the run.

Back to Table of Contents

E. FetchDown &the "To Download" file.

FetchDown uses the "To Download" file to do downloads in just the same manner as FetchUp uses the "To Upload" file for uploads. It ignores files after a blank line, and it writes to the "Activity Log" file after each successful file download. The main difference from FetchUp's procedure is that FetchDown does not touch the changesFile, since it is only updating your local machine, which will in fact just be catching up to the remote site (and to your current copy of the changesFile).

Interrupted downloads, due to a crash or broken connection, are handled slightly differently than in the case of uploads. The reason for this is that broken downloads only affect your local machine and no one else. During the download process, each file is logged, one by one, in the "Activity Log" file after successful arrival. Additionally (if you have not disabled auto-removal), each file is also removed one by one from the "To Download" file. Therefore, if your download is interrupted, all you have to do is run FetchDown again and it will simply pick up from where it left off. (However, if you have disabled auto-removal, you'll have to compare the "Activity Log" file with the "To Download" file and remove the successful transfers manually from the "To Download" file).

If no files are queued for downloading in the "To Download" file, FetchDown quits without logging on and informs you that it was given nothing to do.

After logging on, if FetchDown sees that remote activity has taken place, the download is aborted, and it switches modes to operate as FetchChanges: any new files are added to your downloadList, and you are alerted. You may then readjust your downloadList and rerun FetchDown. FetchDown also checks the site again at the conclusion of the downloads and will again write anything new to the downloadList and alert you if someone else was at work during your download.

If the command key is pressed at (or shortly after) launch, a dialog will appear which asks if you want to override your setting for behavior on completion of the download.

F. FetchTimeStamp, FetchMe.

These two additional applications work by drag-and-drop only (they won't do anything if double-clicked). Both identify and operate on every file in any possible combination of dropped items (files and/or folders - including all nested items down to the last file in the last folder). This means that you may select, by shift-clicking, any desired assortment of files and/or folders, and drag and drop them all at once onto either of these applications; any included files will be handled as if dropped one at a time, file by file.

FetchTimeStamp will set the modification date of all received files to the current time, useful for acquired or moved material which would otherwise be skipped by FetchNewLocal.

FetchMe will add all received files to the "To Upload" file directly, without having to run FetchNewLocal, and without consulting modification dates. It does not add any blank lines to the "To Upload" file, so it may be run multiple times in succession to put together a single unbroken list. Its purpose is for performing immediate uploads or doing small jobs for which running FetchNewLocal is "overkill"- for example when only a few files are worked on, or when all updated files are in a single directory. However, be warned: the author has occasionally made mistakes using FetchMe - forgetting, for example, to upload a graphic associated with a new HTML page. Running FetchNewLocal, although more time-consuming, is always a safer procedure.

Back to Table of Contents


5. The Utilities

The Fetch-O-Matic Utilities reside in a subfolder in the Fetch-O-Matic folder. They may not be moved from this folder, nor may the associated text files be renamed or moved. (If the text files aren't found then they will be recreated, so they may in fact be moved and renamed to preserve them in their current state). There are five utilities: FetchCompare, FetchDelete, FetchMeNot, FetchProfile, and FetchStructure, and four text files which will be automatically created: To Delete Local, To Delete Remote, Unique In Local, and Unique In Remote. Additionally, FetchCompare creates the file "remote home filelist", which is a complete list of every file on the remote site, in the Auxiliary folder.

A. FetchCompare.

FetchCompare compares your local site mirror with the remote site. It first generates a list ("remote home filelist" in the Auxiliary folder) of the contents of the remote site by directory, similar (see note*) to the local list generated by FetchNewLocal/List Files, and then generates two files, Unique In Local and Unique In Remote, which contain the differences. If you find any files in either list which ought to be in both places, simply copy and paste them to either the To Upload or To Download lists as appropriate (i.e., unique local files go into the To Upload file and unique remote files go into the To Download file). There is another thing you might want to do about unique files: remove them. To do this, use the application FetchDelete.

In order to do the comparison, you must of course have a local site filelist. If you have a current one you can save time by using it instead of making a new one. When you start FetchCompare, a dialog box presents the options.

File modification dates are not used in the comparison, because they can be misleading. For example, if you download something from the site it will have a modification date later than the one on the site, because the Mac assigns a new date upon arrival. Nevertheless, all dates are collected and appear in the Remote Site Filelist. You can sort this list (and the local list also) with a text editor or word processor to see everything listed in date order - that's the reason Fetch-O-Matic uses the YY/MM/DD date format.

FetchCompare is quite slow, but behaves itself in the background reasonably well. Its slowness is a result of a) having to open every directory on the remote site; b) the need to interpret and rewrite the information it obtains; and c) the comparison it makes between the local and remote sites. On a site with thousands of files, a complete run could take several hours. To minimize online time, constructing the remote filelist is done in two steps; first the information is collected, then it is reformatted offline. (If your online session is interrupted, the partial filelist will be in a different format than a complete list. This is not cause for concern; the format is not changed until the list is complete).

Because FetchCompare is slow, some options are provided on launch. One option is a choice to only get the remote list and postpone the comparison, or to only do the comparison if you already have the list. The other option is the capability to resume gathering a remote list from somewhere in the middle, if the list creation was interrupted. If this happens, examine the Remote Site Filelist. You will notice that it is in reverse order, bottom to top, with a line with a date followed by a line with a filename. Find out at which item (file or directory) of the root directory it was cut short, then determine the number (up from the bottom of the list) of the incomplete item as it would appear in a Fetch window showing your site root, and enter that number. Your list will be resumed from that point. The mechanism is intelligent and will find the exact point at which the break occurred, so it won't make double entries; it will also quit and tell you if you entered a starting file which doesn't show up on the partial list, with the danger that you will have missed files. If you attempt to run FetchCompare in compare-only mode and your remote list is incomplete or nonexistent, you will also get an informative error message.

FetchCompare does not recognize the existence of empty folders, and neither does FetchMeNot. However, FetchStructure will locate any empty folders on both local and remote sites.

*Note: There is one difference between Mac and UNIX file systems which you may encounter. On UNIX systems, filenames and directory names which contain numbers are alphabetized with the number interpreted as a number: MyFile10 comes after MyFile9. But on the Mac, unfortunately, the opposite occurs: MyFile10 comes before MyFile9, because the digits of the numbers are treated individually as characters and not as part of a number, and 1 comes before 9. What this means is that if you open the remote list and the local list and compare them side by side, then if you have filenames or directory names with multi-digit numbers in them, things will not line up as you expect. Because of this quirk, FetchCompare must do its work by exhaustive, not merely alphabetic, comparison, which is one reason it runs slowly).

Back to Table of Contents

B. FetchDelete.

FetchDelete removes files (and/or directories) from either, or both, the local and remote site copies. When deleting files locally, they are moved to the trash, and if you make a mistake you can open the trash and use the "Put Away" menu command in the Finder to restore them. However, deletion via FTP is not undoable, so you must be extremely cautious with this option.

The operation of FetchDelete is similar to FetchUp and FetchDown. It reads the To Delete files, ignoring anything after a blank line, and will remove deleted items from them if autoRemove is turned on, and it will log everything it did in the Activity Log.

Paste any items you want to trash into these files from the Unique lists, or, alternatively, you may add to these lists (both of them at once) by dragging and dropping local files and directories on to FetchMeNot, see below. You may also manually add to these lists. Use a path only for levels deeper than the root, and use colon delimiters.

When you run FetchDelete, you are first presented with a dialog box to select which site(s) (local and/or remote) you want to remove files from. As a precaution, before doing anything, the list(s) in question will be opened by your text application for confirmation. You may then edit them, but if soy ou must save the changes before continuing. You will then need to select FetchDelete from the Application Menu to proceed. Finally, two more dialog boxes confirm your intentions and allow escape before execution continues.

FetchDelete can remove entire directories. Removing directories removes all files and subdirectories in them as well. Be very, very careful here - you could delete a whole lot of website with a single line. To delete a directory, it must have a trailing delimiter. Just find a path that has the name of that directory in it, and eliminate everything after that directory except the colon. (You may then also erase from the deletion list any files that were in that directory, but this is not necessary. If files and folders are found not to exist, such as after deletion of their parent directory, they will be skipped without complaint).

FetchDelete does not write automatically to the changesFile. This is because the deletions will not involve any further FTP activity. If you have removed items from the remote site, if there are other site mirrors then you should alert their owners of the deletions by using the messageFile. You should paste a list of deletions into this file from your Activity Log, and then the others using Fetch-O-Matic can copy this list to their To Delete Local file and use FetchDelete to remove these items from their local machines.

Back to Table of Contents

C. FetchMeNot

FetchMeNot functions very much like FetchMe; it's a drop-only application which adds all dropped items to the top of both "To Delete" lists. These items can then be deleted from the local and remote sites by FetchDelete. Dropping a folder of items produces a list of files in it, not just the name of the folder, even though it may be desired to delete the entire folder. But it was thought safer to exhibit the entire contents so that you will be reminded of exactly what will be deleted. As noted in the discussion of FetchDelete, you may then truncate the list to show only the parent folder, but this is not necessary.

Unlike FetchMe, FetchMeNot also allows you to drop empty folders and have them added to the "To Delete" lists.

D. FetchStructure

This is a utility to ascertain the directory structure on your local and remote machines. It writes what it finds to a file named "Site Structure <date>" which it will create in the Fetch-O-Matic Utilities folder. You have the option to look at either or both the remote and local sites. FetchStructure examines every directory and identifies any empty ones, which it lists separately at the top of its output file, followed by all the others.

Remote investigations can be resumed as with FetchCompare. You can look at the output file to see where to resume. However, this resumption will overwrite an existing file rather than attempt to add to it. If you want to preserve the partial previous information, rename the file before resuming.

Back to Table of Contents


6. Miscellaneous.

A. "Secret" files/areas: Multiple Fetch-O-Matic suites on one site.

Since Fetch-O-Matic is only aware of files which get listed in its text files, it is possible to have a website with files and/or areas which are independent of Fetch-O-Matic and ignored by it. This may be desired if a particular person or group is responsible for a website portion which stands pretty well on its own, or at least has links to and from another area which don't need to be updated frequently or at all (because they only point to index pages, for example). Not only is this possible and practical, but in fact you can run an entirely separate Fetch-O-Matic suite for this other portion - or several such. All that is required is that the changesFile and messageFile have separate names and/or locations for each "secret" Fetch-O-Matic suite, and that all files in question are not updated by unauthorized persons "from the other side" and thus don't ever appear in their changesFile.

B. Stopping execution.

To stop execution in progress of any Fetch-O-Matic application, you may use command-period; this keystroke won't always work instantly and on rare occasions may produce strange error messages (which can safely be ignored). How well command-period functions depends on which application is running in and what it's doing. A more brutal and instantaneous approach is to type command-option-escape, the universal Mac application "kill" command. In order for either keystroke to work, the Fetch-O-Matic application will first need to be brought to the front. Use the Application menu to do this, then type the keystroke. Killing a running application with command-option-escape may cause instability in your system. You should save any open files and reboot; even though everything may seem healthy enough, you are courting a system crash if you continue, particularly if you try to run the killed application again.

The file transfer situation is a little more complex because killing a Fetch-O-Matic application will not tell Fetch to interrupt a particular file's download or upload in progress; it will only prevent the next pending activity from occurring. If you wish to interrupt an ongoing upload or download, you have to also bring Fetch to the front and tell it to stop. If you don't kill the Fetch-O-Matic application first, if you have the "reconnect" setting enabled, Fetch-O-Matic will reattempt the transfer, since it interprets your manual intervention as a connection error. (As for List Files, to stop it you have to bring it to the front also, then cancel).

Stopped execution, whether by design or accident, can produce unwanted side effects. See the "Couldn't Find..." error topic.

Back to Table of Contents

C. Error messages.

Fetch-O-Matic puts up various messages, and puts its error messages both on screen and into the Activity Log file. Should you crash before you have had a chance to read an error message, you can still find out what it was by reading the Activity Log.

Unless the Fetch-O-Matic application is front-most when it reports an error or finishes its run, its error message or informational message will not appear on screen until you select the application from the Application menu. That menu will blink and there will be a diamond next to the applications' name in the menu, indicating that it wants your attention. The messages will wait indefinitely for your attention, until they are read and dismissed.

"Apple Event Timed Out"

This means that Fetch-O-Matic had to wait too long for something to happen and gave up. This is most likely to occur if a connection was dropped while Fetch-O-Matic was trying to talk to the server (such as negotiating a log-on, but not actually uploading or downloading, which will produce other error messages). This will also occur if Fetch drops a connection but that fact is not correctly reported back to Fetch-O-Matic, more likely to occur with MacPPP/FreePPP than with Open Transport, but always a possibility. This will also occur if you run an application that has an opening dialog to which you don't respond. Eventually, with no activity, Fetch-O-Matic quits.

"Couldn't Find..."

If some expected file is not where it belongs or its name gets accidentally changed, you will see this message, which will identify the missing file. You will have to make a manual correction of this situation (except in the case of a List Files Prefs file missing from the System's Preferences folder, which can be corrected by running FetchProfile, see second paragraph below).

There are a few circumstances, involving either a crash or quitting a Fetch-O-Matic application while it's running, which will result in this situation. In normal operation, certain files (the changesFile, messageFile, and List Files Prefs) are renamed with temporary names as they are compared or swapped with another version. Should that process be interrupted, there will be a file with either ".old" or ".temp" after the name, with or without the original still in evidence. To recover the changesFile, in general it is safe to merely trash the newer one, if it exists, and then remove the ".old" designation from the other one, restoring the situation to what it was before the interrupted application was run, and then run FetchChanges, which will then insure that you have the latest version. This could possibly result in doubled entries in your To Download file, but that is better than missing something.

In the case of a missing List Files prefs file, the best solution is to run FetchProfile and reset List Files prefs; as long as there is still a "List Files Prefs" file in the Auxiliary folder (whether or not there is a ".temp" file there as well), FetchProfile will recover automatically, preserving the original List Files Prefs file that had been in the Preferences folder before this situation arose.

"Error number..."

You may encounter error messages with no description, just a number like "-1055". These may have a number of sources, but usually indicate a network or TCP/IP error, such as a link that went dead during an upload or download.

Back to Table of Contents

"Your local changesFile has data not present in the remote file, indicating a prior problem."

When a Fetch-O-Matic application looks at the remote changesFile to determine if activity has taken place, the test it does is that it first reads the local changesFile and stores the first line of that file. It then reads the remote file and looks for that line in the remote file. If there has been no activity, the first lines will match and the test is done. If there was activity, since the changesFile is written to from the top, the first line in the remote file will have been pushed down with new activity above it. The new activity is everything above this first line which matches your first line. (If you have an empty local file, that is a special case and there is no error - the entire remote file appears as new. If that file is empty too, as when first running Fetch-O-Matic, there is also no error).

If you get this error, it is probably due to a prior failure in uploading the remote changesFile, or other corruption thereof. Whatever the cause, Fetch-O-Matic can have no way to automatically recover in this situation, since it depends entirely on the validity of the remote changesFile. You must then examine your local file and the remote file and try to figure out what went wrong.

If you are collaborating with others in maintaining a website, you will need to contact them because there could be missing data - they might not be aware that the remote file is not up-to-date or accurate, since their local changesFiles may in fact not have any data not present in the remote file.

To correct this situation, first determine what the proper contents of the changesFile ought to be, correct your local file if required, and manually upload that file.

"Not Connected"
"Unable to connect to remote server"
"Unexpected server response"

The most likely error messages you will see are these, generated by Fetch. The first two are self-explanatory. The third message means that an upload or download or remote directory change was attempted and failed because an expected file or directory wasn't found . This could be just a connection glitch or other momentary anomaly, so just try again. Another possible cause is that you had two Fetch windows open to your server, changed their order while Fetch-O-Matic was trying to use one of them, and confused Fetch-O-Matic as to which one it was using. When you open multiple connections to a single server, the windows have the same name, and Fetch-O-Matic can't tell the difference between them. It's best not to mess around in this way, although it's perfectly OK to open a new Fetch window to a different server while Fetch-O-Matic is in action. If you absolutely MUST open a new window to your server while Fetch-O-Matic is running, do so while Fetch-O-Matic is in the middle of uploading or downloading a very large file, so that it will not attempt any new action simultaneously with your other activity.

"Out of Memory"

See under Memory Allocation, below.

"<<script>>doesn't understand the <<event xxxx>> message"

This is the result of a missing Scripting Addition. Scripting Additions are extensions which add capabilities to AppleScript. A number of Scripting Additions are supplied by Apple Computer as part of the basic AppleScript installation, and some of these are required by Fetch-O-Matic. Fetch-O-Matic also requires the third-party scripting additions discussed under "Installation". There is a list of official Apple Scripting Additions appended to this guide.

"That file or directory is unavailable or nonexistent"

This message may be seen in the event of a lost connection to the FTP server. Fetch is not able in this situation to know what has happened, so this can be seen even if you were in the middle of a download when the connection was lost. Therefore, just ignore this message and try again.

"This file may be nonexistent"

You will encounter this message if you have tried to upload or download a deleted file, or if there is a mistake in the filename in the "To Upload" or "To Download" file. You may also (erroneously) see this message if Fetch drops a connection during a session. If the file exists, then just resume your upload or download session.

"Tried to create temporary file..."

You'll see this message if you have a changesFile or messageFile with the ".old" suffix which you recovered from the trash and put back on your site copy, or if you had a crash while downloading (thus not cleaning up). Fetch-O-Matic will abort when it sees this situation and you will have to intervene and decide what to do. The "old" file is your local copy. If this is is the one you want (see about "data not present..."), move the newer file to the trash and remove ".old" from the local file's name.

"Unix or DOS Linefeeds in changesFile"

If you do all file transfers as raw binary when using Fetch-O-Matic, you may get this message, particularly if someone has used another platform to edit this file. As long as it is only edited on a Mac then you won't see this message, unless you have set this file up to be in some other format (as possible with BBEdit and other editors).

Fetch-O-Matic must convert the changesFile to Mac text format, otherwise it can't recognize the items in it properly. You have the option for Fetch-O-Matic to warn you every time it encounters this situation, or to skip warnings in the future. The only side-effect of encountering these linefeeds is that Fetch-O-Matic will run a little slower due to having to perform the conversion.

Back to Table of Contents

"You must install remote access"

This message, or one like it, will appear in the event that your setting for usePPP does not correspond to the software you are using (i.e., you are using Open Transport but your usePPP setting is set to "F", etc.). Adjust the usePPP setting to reflect what is actually in use.

Other (Strange-Looking) AppleScript errors.

These error messages look different from the usual Fetch-O-Matic ones; instead of "OK" you'll just see a "Cancel" button. These messages may or may not make sense. They may be caused by insufficient memory (see Memory Allocation). Some may appear if you interrupt a running Fetch-O-Matic application with command-period, and these can be ignored. Other AppleScript errors may result from some temporary system instability, such as after a forced quit. You may get this type of error if you have not installed a required Scripting Addition. Note that these messages probably won't give you a clue to what's actually the problem. If you get such a message, run FetchProfile and have it check your installation.

If things have been working and you suddenly start seeing messages of this sort, suspect that some portion of Fetch-O-Matic or AppleScript is corrupt, or that you have introduced an extension or other conflict. The first thing to suspect would be the Settings file. If this has been corrupted, there's no telling what might happen. Next, did you install any new extensions or change your system in any way? If problems persist and you're sure that your installation of AppleScript or Fetch-O-Matic - or an extension conflict - is not at fault, please contact the author describing the situation in detail.

D. Memory allocation.

For most Fetch-O-Matic applications, the "factory" memory allocation has been set to 2 or 3mb. This is adequate for most situations. However, this may not be enough to handle a website of 5,000 files or more and you should allocate additional RAM, an additional megabyte or perhaps even more if your site is really huge. You will get an "Out of Memory" error message, or possibly some other spurious error message, if you have an insufficient allocation. Use the "Get Info" command in the Finder to allocate additional RAM to each Fetch-O-Matic application that asks for it. If you have encountered an "Out of Memory" message, AppleScript may have been left in an unstable state and cause a crash next time you run any AppleScript application, whether or not you increased its memory partition. To be on the safe side, you might want to reboot - or at least, save all open documents.

E. Fetch Preferences

It is assumed that the user is familiar with Fetch for uploads and downloads and will set Fetch's preferences appropriately. Due the the many options of Fetch and many possible environments, no attempt was made to have Fetch-O-Matic deal with Fetch's preferences. The only exceptions to this are: 1) Fetch-O-Matic disables Fetch's sign-on dialog so as to not have it interrupt automatic operations; 2) The cache is ignored so that remote file lists are always refreshed; and 3) name changing (encoding file names) is turned off for uploads, as a safety measure.

For correct operation with Fetch-O-Matic, the following Fetch preference should be set: under the "Download" tab, put a check in box " Overwrite files with conflicting names".

Back to Table of Contents


7. Frequently Asked Questions

Q. Every time I run FetchUp it asks me for the location of Fetch, this is driving me nuts. What can I do about this?
A. In your Settings file, you must enter the complete, exact case-sensitive name for the application as it exists on your hard drive, as in "Fetch 3.0.3" or whatever you've renamed it to. The name is not important, but it must match character-for-character the name in the Settings file.

Q. I get an error "<<script>> doesn't understand the <<event xxxx>> message" when I try to run FetchProfile, and then it quits. What's wrong?
A. You're missing an AppleScript Scripting Addition. Read the section on this topic.

Q. I download huge files and sometimes get dumped off in the middle of a transfer. Fetch has capabilities to resume partial downloads. What about Fetch-O-Matic?
A. That's not possible at present because the resumption functions of Fetch cannot be accessed via Apple Events, the controlling mechanism of Fetch-O-Matic. Dave Matthews, the author of Fetch, has promised this capability in a future release. See reconnect for more information on interruptions.

Q. I ran FetchCompare, it saw it crawl through my site, but then it seemed to be doing nothing forever, so I did a force quit. Why doesn't it work?
A. On large sites (3,000+ files), FetchCompare takes a really, really long time (a couple of hours) to compare the remote site to your local copy. Due to subtle differences between UNIX and Mac file ordering, the comparison has to be exhaustive, and since the remote-site list creation is slow anyway, no effort was made to speed this up.

Q. All those little applications are confusing. Why don't you wrap the whole package up in a single application with a menu interface?
A. If there's enough demand, and sufficient numbers of registered users, this could be done.

Q. I'd like to run FetchNewLocal or FetchMe while I'm doing an upload and have it add files to the upload session in progress. Can I do this?
A. Not at present. This capability is being considered for a future release.

Q. I have a problem I can't seem to solve. If I write to you, will I get an answer?
A. Yes. Please include the results of a run of FetchProfile and also any error messages that appear. This will expedite a solution. I like user reports because that's how bugs and errors in the manual get eliminated. Send mail to egray@kagi.com.

Back to Table of Contents


8. Obtaining Software

The Fetch-O-Matic home page is http://www.filmscouts.com/software/fom.html. This page will always have the latest version of Fetch-O-Matic and links to other required software.

In general, the easiest way to find shareware and get updates is to use the search capabilities of the two most user-friendly Mac shareware sources: the Info-Mac HyperArchive at the Massachusetts Institute of Technology (http://hyperarchive.lcs.mit.edu/HyperArchive/SearchForm.html); and Shareware.com (http://www.shareware.com/).

Info-Mac is updated almost daily and maintains a huge archive of software and has abstracts which you can read before deciding to download.

Shareware.com is a search engine and also shows abstracts of the software it finds. One can sometimes find things here which elude a search of Info-Mac. A great feature is that the abstracts are summarized in the search results, which doesn't happen at Info-Mac. If you find what you're looking for, you can choose a download location near you, which may be a good idea if you are not in North America. Shareware.com also publishes "reliability" ratings for the various download sites.

You should always be able to find the latest versions of Fetch-O-Matic, Fetch, Jon's Commands, and List Files at either of these locations. (As noted earlier, type "jons" when searching for "Jon's Commands").

Finally, http://applescript.apple.com/ is the place to go for AppleScript itself and scripting information, scripting software, and links to scripting sites.

Back to Table of Contents


9. Registration and Payment.

Fetch-O-Matic is distributed as shareware. The version you have is the complete package; there is no "lite" or "demo" version, nor will this version expire sometime in the future. You may run each application 30 times as a trial. If you continue to use Fetch-O-Matic after that, you are obligated to pay the registration fee.

Fetch-O-Matic has the following pricing:

1 to 10 single user licenses: $20 per user
11+ single user licenses: $15 per user

A Site License costs $400 (roughly equal to 27 users) and covers all locations for your organization within a 160 kilometer radius of your site (100 miles). One advantage of a Site License is that you do not need to keep track of how many people at your site are using the software.

A World-Wide License costs $1600 and it covers all locations for your organization on the planet earth.

Paying for Fetch-O-Matic is fairly simple. Open the Register program that is in the folder with this file. Enter your name, your email address, and the number of single user licenses you desire to purchase (or Site or World-Wide licenses). Save or Copy or Print the data from the Register program, and send the data and payment to Kagi, a clearing house which handles all payment processing for Fetch-O-Matic.

If paying with Credit Card or First Virtual, you can email or fax the data to Kagi. Their email address is sales@kagi.com and their fax number is +1 510 652-6589. You can either Copy the data from Register and paste it into the body of an email message or you can Save the data to a file and you can attach that file to an email message. There is no need to compress the data file, it's already pretty small. If you have a fax modem, just Print the data to the Kagi fax number.

Payments sent via email are processed within 3 to 4 days. You will receive an email acknowledgment when it is processed. Payments sent via fax take up to 10 days and if you provide a correct Internet email address you will receive an email acknowledgment.

If you are paying with Cash or USD Check you should print the data using the Register application and send it to the address shown on the form, which is:

Kagi
1442-A Walnut Street #392-9A
Berkeley, California 94709-1405
USA

You can pay with a wide variety of cash from different countries, but at present, if you pay via check, it must be a check drawn in US Dollars. Kagi cannot accept checks in other currencies; the conversion rate for non-USD checks is around USD 15.00 per check so this is not practical.

If you have a purchasing department, you can enter all the data into the Register program and then select Invoice as your payment method. Print three copies of the form and send it to your accounts payable people. You might want to highlight the line that mentions that they must include a copy of the form with their payment.

Kagi cannot invoice your company. You need to generate the invoice and handle all the paperwork on your end.

Please do not fax or email payment forms that indicate Cash, Check or Invoice as the payment method but don't contain the payment. (Last time we checked, there was still no technology to transfer physical objects via fax or email, and without the payment the form cannot be processed).

Payments sent via postal mail take time to reach Kagi and then up to 10 days for processing. Again, if you include a correct email address, you will hear from Kagi when the form is processed.

Fetch-O-Matic has a shareware registration reminder dialog which appears after 30 runs. When you pay, you will receive a code to enter in the Fetch-O-Matic Settings file to turn off this reminder. If you do not have an email address, please enter your complete postal address - and please remember that we do not know what country you live in, so please enter that into the postal address also.

If you do not have an email address you should consider selecting the Postcard Receipt so that Kagi can inform you of your registration code. Kagi transmits the registration codes via email and paid postcard receipt only.

Back to Table of Contents


10. Legal Notice.

Fetch-O-Matic is pretty robust and doesn't make mistakes on its own, but it is a potent tool. Improper configuration or other operator error can result in lost data due to the fact that Fetch-O-Matic overwrites existing files. Before using Fetch-O-Matic, and at all times in any event, you should have a backup of your website.

USE AT YOUR OWN RISK. BY USING THESE APPLICATIONS, THE USER ACKNOWLEDGES THAT THE AUTHOR AND DISTRIBUTOR WILL ACCEPT NO RESPONSIBILITY OR LIABILITY FOR LOST DATA, LOST TIME, MENTAL ANGUISH, ILL HEALTH, OR ANY OTHER CONSEQUENCES RESULTING FROM SUCH USE. NO GUARANTEE OF FUNCTIONALITY OR APPLICABILITY FOR A PARTICULAR SITUATION IS GIVEN OR IMPLIED. WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED. CONSEQUENTIAL AND INCIDENTAL DAMAGES ARE EXPRESSLY EXCLUDED.

Back to Table of Contents


Appendix A: List Files settings.

The following screens are accessed from the Options menu in List Files. They are shown here as they should be configured for use with Fetch-O-Matic. (Note: some selections are not relevant, but it's probably a waste of time to try to figure out which ones you can ignore). The final screen, "File Saving Options", has 3 options you should set: the destination folder for List Files' output file, which must be the Fetch-O-Matic Auxiliary folder, the file's suffix (which should be as shown), and the file's creator (shown here as ttxt, the code for SimpleText). If you are using BBEdit, its creator code is R*ch; Tex-Edit Plus uses TBB6, but in any case you can click the "Select" button to choose your text application.

<

Back to Table of Contents

Appendix B: List of Required Official Apple Scripting Additions

The following components should have been installed in your Scripting Additions folder by the System Installer or AppleScript installer (if you are missing any of these, then you will need reinstall AppleScript):

Beep, Current Date, Display Dialog, File Commands, Numerics, Read/Write Commands, Run Script, String Commands.

This is not a complete list of official additions; these are just the ones required by Fetch-O-Matic. It is recommended that you leave all the official additions in place. They are not loaded at startup and will not adversely affect performance, since the operating system is not aware of them until they are called.

Appendix C: Version History, Updating Notes

Note about updates for users of previous versions: There are two new features in v.2.0.3 which add new settings, "reconnect" and "ignoreFolder", to the Settings file. If you first copy these new settings and descriptions from the newly downloaded file into your old Settings file, it is then not necessary to rewrite the Settings file; the old one may be dragged to the new Auxiliary folder, replacing the newly downloaded one. You should similarly also drag your existing "To Upload", "To Download", "Activity Log" (and any other text files you want to keep) to their correct new locations. The new FetchProfile should then be run, and List Files Prefs should be reset (if using List Files) so that the new Fetch-O-Matic folder is recognized. Your old Fetch-O-Matic folder may then be discarded.

V. 2.0.3 (December 28, 1997) fixed all remaining bugs reported in the initial months of release; added two new features, "reconnect" and "ignoreFolder".

V. 2.0.2 (Oct. 26, 1997) fixed a potential problem with non-U.S. and customized date formats (set via the Date and Time control panel), and a small bug which occurred under OS 8 if List Files was already running before launching FetchNewLocal. Also, some error messages were rewritten to be more informative, and there were minor improvements to the User Guide.

V. 2.0.1 (Oct 23, 1997) fixed a bug in v.2.0 which affected certain users, rendering Fetch-O-Matic inoperable on their machines.

Version 2.0: First public release, October 19, 1997

Appendix D: Credits and Thanks

Special thanks to Jeremy Posner.

Thanks to Mark Alldritt for allowing free distribution of the MacPPP Control Scripting Addition.
Thanks to Mikael Flensborg for permission to distribute the setDate Scripting Addition.
Thanks to Sebastian Kloska for permission to distribute the FreePPP Control Scripting Addition.
Thanks to Jim Matthews, author of Fetch, for making it so complete, solid, and scriptable.
Thanks to Alessandro Levi Montalcini for List Files and other great shareware.
Thanks to Jon Pugh for Jon's Commands and many other contributions to the Mac community.
Thanks to Apple Computer for providing the scripting environment to make all this possible.
Thanks to all the generous contributors to the Mac Scripting Systems list, an invaluable resource.

Thanks to my two cats, One-Many and Too-Many, who often reminded me to eat.

Copyright © 1997 Webcast Multimedia, Inc. All rights reserved
Fetch is a registered trademark of Dartmouth College.

Back to Table of Contents